Function from visual analysis and physical interaction: a methodology for recognition of generic classes of objects

This paper presents an overview of the GRUFF-I (Generic Recognition Using Form, Function and Interaction) system, a nonpart-based approach to generic object recognition which reasons about and generates plans for interaction with three-dimensional (3D) shapes from the categories furniture and dishes. The system operates as follows. A researcher selects an object and places it in an observation area. An initial intensity and range image are acquired. These are input to a three-stage recognition system. The first stage builds a 3D model. The second stage receives as input a 3D model and considers the shape-suggested functionality of this shape by applying concepts of physics and causation (e.g. to infer stability) to label the object’s potential functionality. The third stage uses this labeling to instantiate a plan for interaction to confirm the object’s functional use in a task by incorporating feedback from both visual and robotic sensors. Results of this work are presented for eighteen chair-like and cup-like objects. Major conclusions from this work include: (1) metrically accurate representations of the world can be built and used for higher level reasoning; (2) shape-based reasoning prior to interaction-based reasoning provides an efficient methodology for object recognition, in terms of the judicious use of system resources; and (3) interaction-based reasoning can be used to confirm the functionality of a categorized object without explicitly determining the object’s material composition.